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ABSTRACT 

In this study, we develop an interactive AR Learning System based on Augmented Reality and interactive 
touch-screen. The learning content knowledge is about conservation of fish in Taiwan. The system combines the 
game by the concept of AR book which allows children to learn about the importance of conservation of fish. A 
mechanism is designed to verify whether the game based on AR book well delivers the educational messages. 
The whole mechanism is an interactive game using touch screen to interact with the rod as a tool to get rid of the 
shackles of keyboard and mouse. It provides a more realistic interaction. The evaluation results show that the 
interactive AR Learning System has positive usability. 

INTRODUCTION 

The concept of conservation is popular presently in Taiwan. Therefore, we must come to understand and learn 
about the basic information of fish. There are six kinds of Taiwan endemic species fish which have been extinct 
and nine kinds of endangered Taiwan endemic species fish (Baskin, 1994; Wang, 1999). The cause of impacts 
includes ecological changes in habitat, alien species invasion, river fish poisoning and electrocute, and so on. 
The rivers have been serious polluted and damaged by human’s illegal hunting, abandon alien species, land 
development without planning, and so on. It causes endanger of Taiwan endemic species of freshwater fish. 
Conservation of fish is the responsibility of all people. The river conservation action must be taken from daily 
life. 

Augmented reality (AR) has been increasingly applied in various fields. Many studies use augmented reality in 
the design of instructional systems for investigating issues such as learning stimulus, motivation and 
achievement. We develop a conservation species learning system through the AR technology. In AR, through the 
combination of virtual objects and real scenes, learners are able to interact with virtual fishes, getting the sense 
of excitement to improve their learning effects and interests. 

LITERATURE REVIEW 
Augmented Reality 

AR is a new technique of the computer vision application used to facilitate interaction in the digital arts. 
Recently, many scholars and institutes have conducted research examining AR, which is also called Mixed 
Reality (MR) given that it is an extension of Virtual Reality (VR). Using computer graphics, VR can simulate 
objects in the real world and create an environment in which people can interact with the simulated objects. AR 
is the image, object, or scene that is generated by a computer to blend with the real environment to enhance the 
visual experience. To summarize, AR adds virtual objects to the real environment. AR technology must possess 
three characteristics: the combination of virtual objects and the real world, real-time interaction, and the 
representation of 3D space. 

Milgram and Kishino (1994) treat the real environment and the virtual environment as a continuum, as shown in 
Figure 1. The real environment is on the left end and the virtual environment is on the right end. VR typically 
replaces the real world, whereas AR augments the virtual images produced by the computer with objects from 
the real environment. Presently, AR is applied extensively in the fields of education, medical technology, military 
training, engineering, industrial design, arts, and entertainment (Azuma, 1997; Azuma, et al., 2001). 

AR combines virtual objects with the real environment and displays the virtual objects generated by computers 
to users. Milgram and Kishino (1994) define two ways of displaying AR: See-Through AR and Monitor-Based 
AR. In See-Through AR, the users can see the surrounding environment through a monitor that also displays the 
virtual image. Accordingly, the effect of the augmented environment is strongest with See-through AR. In 
Monitor-Based AR, a computer combines images captured by a webcam with virtual images. The final image 
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after this combination is displayed on a Head-Mounted Display (HMD) or on a computer monitor. HMDs are 
either pure or equipped with a small webcam. The former system is small and can be equipped with a 
head-mounted tracking instrument that tracks the viewing angle and the direction the user’s head is facing. This 
pure HMD is more suitable for research and for the application of AR. The HMD with a small webcam has an 
immersion effect (Hsieh & Lin, 2009; Hsieh & Lin, 2010; Hsieh, et al., 2010). 

Now, plenty of scholars apply AR to education, Diinser and Hornecker (Diinser & Hornecker, 2007) observes the 
condition of students’ reading AR textbooks, and further probes into how students from 6 to 7 years old operate 
the novel interactive teaching media. When students use paddle to interact with the AR textbook, the monitor 
will display text, sound, avatar etc. related to AR textbooks. The main focus is on that we can observe students’ 
reaction while they are using touchable interactive interface. Liarokapis et al. (Liarokapis et al., 2002) propose a 
Multimedia Augmented Reality Interface (MARIE) E-Learning, applied to engineering education, in order to 
enhance traditional teaching and learning methods. The AR system uses virtual multimedia to interact with users 
and is fitting for the tabletop environment. MagicBook (Billinghurst et al., 2001) exploits AR to design a set of 
books for user to read. It is like a normal paper book, but specially, the content in the book is made up of 3D 
animation, presented by AR, which shows the information from the book. A book is a real object while its 
content is a virtual object. Therefore, users can use handheld HMD to experience the AR scene in Magic Book. 
This reading method can help users turn their imaginary world into reality and then inspire more imagination in 
reading. 

Touch Screen Technology 

There are several types of touch screens: resistive touch screen, capacitive touch screen, SAW touch screen, 
optical touch screen, and electromagnetic digitizer. In this study, we use optical touch screen as system 
interactive interface. The comparison of touch screens is to be delivered in the subsequent section (Sears, 1991; 
GTouch Groovy Technology, 2011). 

Resistive touch screen and capacitive touch screen: The advantages include fast response and high accuracy of 
recognition. But, its reflectance is higher and the material fatigue easily. Therefore, it cannot last long life to use. 
Capacitive touch screens are specifically designed to improve the shortcoming of low scratch resistance with 
resistive touch panels. 

Optical touch screen and SAW touch screen: Such kinds of screens can be easily installed and are not affected by 
the panel size, but it is non-linear combination of distortion. Hence, the software must calibrate so that the 
response speed is slower. SAW touch screen technology overcomes the flaws in capacitive touch screens that are 
susceptible to interfere by signal noise and static electricity. 

Electromagnetic digitizer: Electromagnetic digitizer operates based on electromagnetic sensing; it involves the 
use of an electromagnetic pen that functions as a signal transmitter while the electromagnetic board acts as a 
receiver. It is stable, reliable, high-accuracy rate, low reflectivity, locating accuracy, and movement sensitivity. It 
belongs to a kind of linear structure of the touch panel. 

System Development 

Augmented reality enables users to see the real world with virtual objects superimposed upon it. In this study, we 
develop an Interactive Augmented Reality Learning System (IARLS). The system is divided into two parts: 
hardware and software. In the hardware the touch screen is used and in the software Virtools Dev 4.0 and 3D 
Max 9 are used. As to the implementation of augmented reality, we combine ARToolKit and Virtools. Figure 1 
shows the system development tools. 
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Figure 1: System development tools 


In this work, several avatars are designed, including conservation of fish, alien species fish, and river pollution 
objects, as describe below. 


(1) Conservation of fish: We design three kinds of conservation of fish to let learners select in the system of 
learning game. There are Oncorhynchus masou (see Figure 2), Macropodus opercularis (see Figure 3), and 
Varicorhinus alticorpus (see Figure 4). 

(2) Alien species fish: The river brings in alien species from elsewhere is one of the major factors that causes 
conservation of fish to die. It includes Gambusia affinis (see Figure 5) and Oreochromis mossambica (see 
Figure 6). 

(3) River pollution objects: They indicate the factors that harm the environment. They are discarded fish hook 
(see Figure 7) and garbage in system. 



Figure 2: 

Oncorhynchus masou 



Figure 3: 

Macropodus opercularis 



Figure 4: 

Varicorhinus alticorpus 




Figure 5: 
Gambusia affinis 


Figure 6: 

Oreochromis mossambica 



Discarded fish hook 


The system development contains five steps. 

• Step 1: Building avatars, objects and 3D scenes via 3Ds Max. 

• Step 2: Drawing textures of avatars, scenes and system surfaces via Potoshop CS3 and Illustrator CS3. 

• Step 3: Creating the system introduction video via Media Studio Pro 8. 

• Step 4: System programming of augmented reality based on Virtools SDK and ARToolkit as the system 
development environment. 

• Step 5: Finally, we utilize Virtools as system building platform to integrating all components. 
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System Operation 

In the system operation processes, at first the Learners must understand the domain knowledge of Taiwan 
endemic species of freshwater fish via the AR Book. The teaching video is displayed upon the AR Book. The 
system forces learner to learn; the game start only when the learners complete the whole learning processes via 
the AR Book. The game employs a bonus mechanism to give rewards to the learners. As to the 3D model objects, 
there are Taiwan endemic species of freshwater fish, alien species fish and garbage. The alien species fish and 
garbage endanger Taiwan endemic species of freshwater fish. It means that the alien species fish and garbage are 
hazards. Consequently, learner needs to use fishing rod to angle those hazards. The game will increment the 
bonus point when the learner uses fishing rod to angle them. Oppositely, the game will subtract the bonus point 
when the alien species fish and garbage endanger Taiwan endemic species of freshwater fish, or the learner 
carelessness angle Taiwan endemic species of freshwater fish. When the bonus points exceed the threshold, it 
indicates that the learner successfully pass the game level. The system operation flowchart is depicted as in 
Figure 8. The Figure 9 shows the sketch of Taiwan endemic species of freshwater fish in the AR Book and the 
system installation is shown in Figure 10. 



Figure 8: System operation flowchart 



AR marker 


Avatar-Taiwan endemic 
species of freshwater fish 


Webcam —^ 

0 


Figure 9: The sketch of Taiwan endemic species of freshwater 
fish in AR Book 



Figure 10: System installation 


The peripheral hardware of the system operation includes webcam, monitor, fishing rod, and touch screen. The 
webcam is used to capture the marker of AR Book. The monitor is used to display the game scene. The fishing 
rod is used to angle hazards by learners. The touch screen shows virtual river, and there are hazards in it, the 
learner can interactive with it by fishing rod. There are two markers under the touch screen in order to locate 
avatars in the virtual river. 
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Figure 12: System operation of the subject 
using fishing rod 



Figure 11: System presentation 



System Evaluation 

In the part of system evaluation, it mainly evaluated the system usability from the end users perspectives. We 
utilized the well-known questionnaire System Usability Scale (SUS) to evaluate the system usability. The 
questionnaire is revised by experts with significant experiences in the related fields. A 5-point scale ranging from 
1 as strongly disagree to 5 as strongly agree is used for the measurement. The revised version of the SUS 
questionnaire is in Table 1 (Brooke 1986, Tullis and Stetson 2004). The revision majorly focused on making 
SUS more suitable to system evaluation. 

SUS is a questionnaire to evaluate users’ subjective impressions about the system and their degrees of 
satisfaction. In the aspect of system usability evaluation, the SUS is an efficient, time-conserving, and 
labor-saving way of subjective assessment. At present, it is widely applied in the system usability. After users 
finishing answering the ten questions, the scale offers a formula which transfers the subjective impressions of 
users into the objective data information for analysis. That is, the score of SUS is used to evaluate usability of 
the system. The range of estimate score is from 0 to 100. The higher the score is, the more useful the system is 
and the more easily users can interact with it (Brooke, 1986; Isman & Isbulan, 2010; Liu & Lin, 2010). 

_Table 1: SUS questionnaire_ 

_System Usability Scale_ 

1. I think that I would like to use this system frequently 

2. I found the system unnecessarily complex 

3. I thought the system is easy to use 

4. I think that I would need the support of a technical person to be able to use this system 

5. I found the various functions in this system are well integrated 

6. I thought there is too much inconsistency in this system 

7. I would imagine that most people would leam to use this system very quickly 

8. I found the system very cumbersome to use 

9. I felt very confident using the system 

10. I needed to learn a lot of things before I could get going with this system 


Subjects finish operating the system and fill out the SUS questionnaire. Listed in table 2 are the “SUS scores” of 
the interface of the IARLS system. As summarized in Table 2, the mean SUS score is 78, the median is 66, the 
maximum is 89 and the minimum is 57. These scores indicate that the IARLS system is usable. We have 
interviews with these subjects on their ideas about IARLS after they finish operating the system and filling out 
the SUS questionnaire. Subjects said that this idea is very foresight. The system may be less than perfect and that 
is not too much learning effectiveness. Some of the learners themselves had been used relevant augmented 
reality system. The system could let people understand basic concept of conservation of fish in Taiwan and 
achieve learning by doing. 


Table 2: SUS scores descriptive statistics 



N 

Mean 

Median 

Min 

Max 

SD 

Stat 

33 

78 

66 

57 

89 

11 
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Table 3: SUS Questionnaire and Statistics of Each Item 



System Usability Scale 

Mean 

SD 

1. 

I think that I would like to use this system frequently 

3.37 

0.72 

2. 

I found the system unnecessarily complex 

2.28 

0.77 

3. 

I thought the system is easy to use 

3.81 

0.48 

4. 

I think that I would need the support of a technical person to be able to use 
this system 

2.87 

0.96 

5. 

I found the various functions in this system are well integrated 

3.42 

0.72 

6. 

I thought there is too much inconsistency in this system 

2.64 

0.75 

7. 

I would imagine that most people would learn to use this system very quickly 

3.52 

0.56 

8. 

I found the system very cumbersome to use 

2.41 

0.43 

9. 

I felt very confident using the system 

4.01 

0.41 

10.1 needed to learn a lot of things before I could get going with this system 

2.08 

0.68 


Beside the SUS score that shows the usability of IARLS, this survey in Table 3 reveals the following findings: 

• The 9 th item gains the highest mean. It indicates that most learners feel very confident while using the IARLS 
system. 

• The next four with higher ranks are the 3 rd , 7 th , 5 th , and 1 st items. They showed that this system achieves good 
characteristics such as easy to use, quick to learn, well-integrated, and attractive to interact with. 

• The standard deviation of 4 th item is relatively high. It indicates that some learners are familiar with the AR 
system before attending the experiment, but some are not. Therefore, some needs technical supports, but 
some can interplay with the system totally by herself/himself. 

Subjects give some comments for us that it will be able to enhance learning effectiveness for learners if the 
system of the fluency is more smoothly and more stable. And so, there are some reasons we must improve for 
system through analysis of SUS questionnaire as follows. 

• The system is a little bit complicated: Subjects feel that the procedure to use the system is a little bit 
complicated. Subjects who operate system need taking AR Book and look at the teaching video at the same 
time. Then, subjects must pick up the fishing rod and play the game. Although it leads to a simple and 
interesting learning system, but the procedure is complicated. 

• The system is not stable enough: The system is not perfect. Sometimes it causes crash occasionally during 
the operating procedure. If the system crashes, it must be restarted and so that the learners must re-learn again. 
For this reason, Subjects feel some trouble. 

• Need the assistance of technical staff: Although subjects have taken information technology and e-leaming 
courses, they have only a handful of contact with augmented reality. However, there are some subjects who 
do not know how to manipulate the system and AR Book. Thus, we will add the operation manual or 
instructions that users read it before they manipulate the system. 

CONCLUSION 

In this paper, we propose an augmented reality system with an interactive touch-screen technology in an 
innovative way on the learning of Taiwan endemic species fish. The environmental consciousness is popular 
currently. Therefore, this study aims to make people have a better understanding of Taiwan fish conservation 
issues. The system is well designed and evaluated via SUS survey and interview. The evaluation shows that 
IARLS promising. 

Based on the above two evaluations including questionnaires and interviews, the evaluation results show 
that this system achieved positive usability, the learners enjoy the interaction with the developed system, and 
most significantly, such kinds of AR systems are acceptable. Listed below are some results and findings after the 
evaluation. 

• The AR interfaces of IARLS system are easy to use and not complex. 

• This system is easy to learn and instructive to derive imagination. Moreover, the learners didn't need the 
support of a technician to help me use this system. 
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• The learners are willing to explore the systems, and gain more comprehension via the AR interface. 

• The system and the AR interface are well integrated. 

• The interface is very user-friendly and made learners feel confident.) 

• The learners would like to interact with this system, and think that the AR interaction is quite attractive. 

• The learners feel quite excited when facing the AR interface. Therefore, they feel emotionally fulfilled, 
mentally satisfied, and have good feelings when interplaying with the ARt-based interface. 

• In sum, the learners feel the AR interface of IARLS is usable, (the mean of SUS score is quite high: 78) 

The results of experimental evaluation of the questionnaire and interview reveal that the learners think that the 
interface is very interesting and novel which they have never touched before. It means that the learners accept 
this kind of interactive interface. Anyway, there are still some directions for improving the system. In the future, 
we will continually revise this work based on the feedbacks of subjects, and perform expert evaluation on the 
system. 
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